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SECTION I, INTRODUCTION 



Using compu'tera for Instructional purposes in developmental 
education is obviously a very timely and confusing topic. Many 
of us are told or feel we muat select some computer hardware 
(the equipment) and/or software (the prograns) and use it. We 
aren^t clear on what^s available^ how to choose^ or hov^ to use 
it. Wa suspect that making the wrong choices at thia early 
stage may blight future use of computers in our programs, but 
that doing nothing ;»ay prove equally as harnf ul . 

This publication is not intended to solve the entire problem. 
It tackles a single piece of the puzzle: how to evaluate 
microcomputer software for instructional use in developmental 
programs. Moreover, it is a primer rather than a treatise it 
provides a short, readable introduction to the topic. It ia a 
first edition, to help out NOW, rather than the final word. 

The focus ia on the developmental educator who, without 
training, much knowledge or experience, or noticeable hi^lp, muat 
decide what software to buy. It assumes that you will be an 
island unto yourself, and that you are not well-veraed in wisely 
selecting microcomputer software. It assumes that you already 
have the hardware. It assumes that you are willing to take the 
time to makc^ software si^lectlon a systematic rather than a 
random decision. 

Evaluating software doea not exist in a vacuum. In making a 
deciaion we need to conaider at least four ingredients: the 
typea of software there are, the pertinent characteriatica of 
developmental studenta, the crucial character istica of 
microcomputers;, and the inatructional purposes we have in mind. 
Sections II-IV of this document deal with the first three 
ingredients in the software aelection equation. Section V 
presents a general evaluation process, while Section VI itemizes 
an evaluation form for you to modify. Section VII discusses 
previewing of software, while Section VIII liata some 
particularly useful sources of published software evaluations 
and a few references. Section IX closes with some suggestions 
of further things which NADE is considering doing in the area of 
microcomputers in developmental education. 



SECTION II. TYPES OF MICROCOMPUTER SOFTWARE 

Microcomputer software falls into at least six groups, 
distinguished mainly by what it does, and how it ia used. 
Distinguishing among them is important when deciding if a 
particular piece of software is worth purchasing. The types can 
usefully be divided by differencea in what they ask the computer 
to be and do. 
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A. COMPUTER AS TEACHER 

Two of the alx typea mainly ask the computer to teach. 

1. Drill & Practice, This moat common software type 
aervea to reinforce previously- learned akllla • It 
preaents a controlled ^ though hopefully 
randomly-generated^ set of exerclsea of activities, 
often within a gaming format. It should contain several 
difficulty levela , and large numbers of problems . It 
works best with narrow p wel 1- specified skills, and 
sometimes tracks student performance. It can become 
boring and uncreatlve, or the gaming aspect can 
overwhelm the learning Intentxon • 

2. Tutorial * This type presents complete Instructional 
units of new Information, usually without help from the 
Instructor . It can Initial ly teach something , or can 
reinforce, enrich , apply, or broaden something whose 
rudiments have already been taught • It works best with 
clearly-defined basic concepts and strategies, when 
there Is relatively high learner motivation. It should 
contain several levels of d If f Iculty , should diagnose , 
and should then adjust the material to Individual 
needs/abll Itles . A tutorial program can bc^come very 
dreary If It just presents text, arid then asks questions 
for comprehension. A danger Is that a student, once 
lost or discouraged, will have difficulty getting back 
on the predetermined track. Too often, (s>he can get 
back to where (s)he stopped only by repeating exactly 
what has already been studied. 

B. COMPUTER AS MODEL 

A second, broader category of software asks the computer to 
create an artificial model of some aspect of reality, and then 
to have the student act with and within It. 

3. Simulations & Games . This type of software creates 
real or Imaginary physical or social realities (fur 
traders, factories, space colonies, fantasy or adventure 
worlds, etc.), and asks the student to make choices and 
then deal with the consequences. It calls for heavy 
student Involvement ("Interactive"), with a strong 
emphasis on problem-solving. It should provide hints 
and clues to help students discover critical points, but 
can also often be very open-ended, require both thinking 
and doing and/or allow several paths to mastery. A 
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piece of game software rarely directly teaches anything 
other than problem -solving^ but can be the creative 
vehicle for teaching other things (e.g. history, 
vocabulary > . 

C. COMPUTER AS TOOL 

The final three types of software use the computer mainly as a 
tool for doing things. These can be used by the teacher, the 
student, or both. 

4 . Management/ Administration • Microcomputer software 
can be a powerful instructional aid in such areas as 
record- keeping, grade tracking and calculation , 
diagnosing and assigning supplement.al materials and 
activities, generating tests and worksheets, and the 
like. Host of this software is specialized, and not for 
student use. Unfortunately, most of it can only be 
evaluated by using it heavily over a period of time, to 
discover strengths and weaknesses, and after you have 
purchased it. 

5. Operational . There is much software which is an 
operational tool, in three areas: word processing, 
data-based management (manipulating a lot of 
information) , and numerical spreadaheets (for financial 
and statistical calculations) • Any can be a powerful 
classroom t.ool (e.g. teaching writing with a 
word-processing program, or research with a data-based 
management program). Such uses depend on finding 
software which is unsophisticated enough for rapid 
student mastery. There are also specialized operational 
software programs (e.g. spelling-checkers, math 
algorithms) which may have instruct.ional implications. 

6. Programming/Authoring . The final type of software 
comes in three versions. Special languages (such as 
LOGO) simplify and often make more creative the writing 
of computer programs. Authoring software (such as Super 
Pilot) allow instructors to write t.heir own special 
computer programs (usually drill and practice or 
tutorial ) without knowing how to program at all . The 
special program itself translates general intentions 
into programming language. And, third, there is 
software which can add graphics, sound, etc. to 
locally -developed programs. 

A given piece of software may combine more than one of these 
types, but can normally be identified as being mainly one or 
another. Choosing the type of software you want, (within the 
context of how and where you plan to use it) is an important 
early decision in your search . 
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SECTION III. CHARACTERISTICS OF DEVELOPMENTAL STUDENTS 

The aecond critical Ingredient. In the software selection mix la 
the particular characterlat lea of developmental atudenta. Few 
aof tware evaluatlona pay more than superficial attention to the 
precise type of student the program will be effective with. 
This dimension la obviously o£ great Importance when we combine 
adult students, K-12 ski lis , and major learning and cognitive 
difficulties. What will help bright, middle-class junior high 
schocrl atudents maater writing skills may well NOT work with our 
studenta. The character iatlcs of poatsecondary developmental 
sttadenta which seem of relevance can be generalized into three 
groups : leorning, academic , and cognitive characteristics • 

A. LEARNING CHARACTERISTICS , 

Developmental atudenta tend to have short attention apans^ be 
easily distracted^ have poor aelf -concepts aa learners (believe 
thay can^t)^ pay little attention to detalla and directions^ are 
not aelf -start ing or aelf -motivated^ have Inadequate learning 
skills^ and need major teacher and peer assistance or presence. 

B. ACADEMIC CHARACTERISTICS , 

Many of them have poor comprehenaion akllla, poor vocabularlea, 
very narrow background knowledge (hlatory, culture^ aoclety, 
etc • > , short memoriea and retention, 6tc. 

C. COGNITIVE CHARACTERISTICS , 

They tend to be: rigid, concrete thinkera; unaware of how they 
or anyone elae thinka; one-ahot problem-aolvera, reluctant to 
take riaka; limited in ability to generalize from examplea, 
analogiea, etc.; and almoat totally reliant on the loweat levels 
of cognition (memory) • 

Precious little software is written for this population^ though 
there la some writtien for general audlencea which can work with 
developmental atudenta. The trick la to detach ourselves from 
our own learning akllla and abilities^ and from the glamour of 
the technology, and view software as our students will. 

SECTION IV, PERTINENT MICROCOMPUTER CAPABILITIES 

The computer, like any technology, haa specific strengths and 
potentials (aome of which we are just beginning to unravel) • 
The major error with much software is that it uaea the compvjter 
aa a mechanical textbook or a mindless fireworks display. Good 
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software bullda deliberately upon thoae things which the 
computer doea better than any other technology. The third 
Ingredient In the software selection equation Is thus those 
pertl nent microcomputer capabl 11 ties • 

A. TEACHING/LEARNING ^STYLE" . 

Computers can give students a sense of control, can reduce 
pressure and Increase privacy (by not having others notice their 
errors) , and can be endlessly patient and consistent. They can 
also force the student to be specific (when students discover 
thtt "you know what I meant" doesn't work with a blind and dumb 
machine), and can begin eroding their habit of meeting 
difficulties by "psyching out'" the Instructor . Major obstacles 
to the teaching/learning exchange can change when students work 
with computers. 

B. RESPONSIVENESS - 

The computer can give startllngly fast feedback, can generate 
endless activities, can constantly evaluate student progress, 
and can quickly change the learning events to respond to student 
troubles or successes. 

C. MULTISENSORY - 

The computer can add pictures and sounds to text, can use 
graphics to replace or highlight text, and can In general 
replace monomodal learning with orchestrated mult Isensor y 
experiences, cheaply and fairly easily. 

D. ACTIVITY . 

The computer can demand continual student Involvement: both 
physical and mental activity. It can outwalt the student, and 
can constantly seek to put the student In a position where he or 
she must do the "making sense" out of the material. 

The computer can do all of these things. Good software will do 
many of them. 



SECTION V, EVALUATING SOFTWARE 
A. EVALUATION PROCESSES 

Adequate evaluation of software takes time and insight. 
Ideally, evaluations will be done by campufi committees , armed 
with detailed forms, and able to spend several hours each at the 
task. In practice, evaluation will often ba done by a single 
harried Individual , with no quality time available and often 
with little Idea of what to look for. 
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Thla Section trlea to provide some in3ight.a into what to look 
for. Each institution^ however^ must build its own evaluation 
process around candid answers to three critical questions: 

« How much time can you spend evaluating a piece of 
software? 

« How^ where^ and with whom will you use the software? 

« Where aro your critical gaps in teaching & 
learning^ which cannot be filled by other means? 

Frank answers to these questions can be the basis for a 
practical local evaluation process. The following two-cycle 
evaluation process is workable^ and adaptable to many types of 
local constraints. 

!• Selection of Possibilities * Identify several pieces 
of software which might meet your needs, and which you 
can obtain for previewing. To do this, scan 
commercial/public software reviews, talk to colleagues, 
invite vendors in to display their wares^ write to 
experts, or visit other institutions which are farther 
down the road than you are. Order several of the most 
likely possibilities, for xeview. 

2. Initial Screening . Subject each piece of software 
to an initial screening, to see if more laborious 
evaluation is warranted (many times, only a cursory look 
will serve to eliminate software). Develop a short 
evaluation form, summarizing your top 8-10 priority 
requirements. If possible, have at least 3 people 
evaluate the software (less than an hour each) . Compare: 
notes, and decide whether or not to proceed with Step 3. 

3. Full Evaluation . A full hands-on evaluation might 
include the following: 

a. Prepare a detailed evaluation form, such as the 

sample later in this section, but tailored to 
your own constraints and priorities. Be sure to 
leave room for qualitative, impressionistic 
comments: don't oust evaluate quantitatively. 

b . Run one part of the software as a successful student 

would. Use right answers, variations on 
answers, attempts to skip ahead, and in general 
challenge the beat in the program . Note even 
minor things that bug you. 

C. NADE 1984 
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c. Run another part as an unaucceasf ul student would, 
with persistent errors, typing mistakes, 
accidentally pushing ESC or RET, etc. Try to 
get confused. Try to start over. Note what 
happens and doesn ' t happen . 



d. Try at least one segment (not at the beginning) 

.xiany times, with all sorts of input, to test all 
the possibilities . 

e. See what the software actually does; don't assume 

that it does something simply because that aeema 
logical , or implied, or implicit . 

f . Have aome atudenta evaluate the software, or 

have them use it and watch what happena. 

4. Compariaon and Deciaion . The final atep ia aimply 
to take the data, and chooae to buy or not. 

a. Compare notea with othera evaluating the aoftware 

on campua, with uaera elaewhere, and with 
publiahed reviewa. If there are major 
diacrepanciea, review the program again on 
apecific pointa. 

b. Make a deciaion about purchaaing the aoftware. 

A couple of final precautiona may be in order. Firat, don't 
evaluate different typea on exactly the same baaia: for example, 
drill and practice aoftware, and tutorial aoftware, ahould have 
different character lat ica . Second, don't get aeduced by the 
technology; work with a piece of aoftware long enough that the 
glamour fadea, and the real quality (or lack of it) comea 
through. Third, never unreaervedly accept what the publiaher'a 
literature or vendor aaya; they will almoat certainly not know 
developmental atudenta (and often will not know the aoftv]»are 
either) . 



B. CHARACTERISTICS OF QUALITY SOFTWARE 



Software tends to be ''quality" if it teachea developmental 
atudenta apecific thinga aa or more effectively than other 
available mean£^ of inatruction, without negative aide effecta. 
Some general characteriatica which promote auch outcomea are the 
following • 



o Suited to developmental atudenta (academically, 
culturally, etc . ) 

o Interactive (continually requiring action, thought, 
choicea) 
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o Baaed on sound teaching/learning theory 

o Embodying sound Inatructlonal practiced 

o Error free (technical, spelling, content, etc.) 

o Easy to uae (for student and instructor) 

o Maximizing the computer's uniquenesses (graphics, 
memory , etc • ) 

o Responsive to individual status (diagnostic, 
branching, etc. ) 

o Building self-concept and reasoning skills along with 
content learn ing 

o Learner-directed and controlled 

Little aoftwere will achieve all of this, and that which cornea 
close will probably be among the most expensive. Nevertheless, 
we need to continually remember what software can be as we try 
to decipher what a particular piece of software actually is. 



SECTION VI. SOFTWARE EVALUATION FORM 

Figure 1, on the following pages, contains a "shopping list" of 
factors that you might want to consider when evaluating 
software. The liat is too long, yet does not include everything 
that someone might find pertinent. We have attempted to build 
from the perspective of a practicing developmental educator, 
trying to find software which will help students while not 
making her or hia job harder. 

YOU SHOULD TAILOR THIS EVALUATION FORM TO YOUR OWN PRIORITIES 
AND CONSTRAINTS. CUT IT DOWN TO MANAGEABLE SIZE, TO Whf«7'S 
REASONABLE FOR YOUR SITUATION. 

The rest of this subsection expands on the items in th^.j rating 
form . 

A. INSTRUCTIONAL TARGET 

In what ways does this program match the c aracteristics of 
developmental students? 

1 . Academic Prerequisites ? Reading, math, spelling, 
and writing skills? Specific areas of academic 
' knowledge? 

C. NADE 19R4 
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DEVELOPMENTAL SOFTWARE EVALUATION FORM 



RATING? 
A. 



INSTRUCTIONAL TARGET 

Approprlaiie for developmeniial aiiudeniia re : 
1. Acadeitlc prerequlaliiea? 
2 • Background /or 1 entai: ion ? 

3. Cognitive akilla? 

4. Motivation/attitude? 

5. Eaae of operationa? 



B. EDUCATIONAL DESIGN 

1. Baaed on aolid teaching/learning theory? 

2. Purpoaea and objectivea clear? 

3 • Content appropriate p accurate^ biaa-f ree? 

4. Stimulatea thinking and creativity? 

5. Inatructional aequence logical p modular? 

C. INSTRUCTIONAL DELIVERY 

1. Keepa atudenta active? 

2. Handlea wrong anawera effectively? 

3. Handlea mechanical errora effectively? 

4. Input methoda clear^ aimple, conaiatent? 

5. Adjuata to individual needa, atrengtha? 

6. Reaponaea build aelf-concept? 

7. Studenta can move about freely in program? 

8. Directiona & examplea clear, uaeful, frequent? 

D. AFFECTIVE IMPACT 

1. Graphica, aound enhance learning? 

2. Intereating and motivating? 

3. Screen deaigna effective? 

E. INSTRUCTOR USE 

1 • Minimum learning, preparation time? 

2 . Inatructor can adapt program content? 

3. Uaeable with varied atudent groupinga, contexta? 

4 . Recorda atudent performance? 

5. Can aerve aeveral computera aimultaneoualy ? 

F. SUPPORT ELEMENTS 

1. Operating documentation helpful? 

2. Supplemental materiala appropriate? 

3. Worka properly? 

4. Uaea available equipment? 

5. Free from errorG ( ape 1 ling, buga, etc.)? 



RATING SCALE: 



Excellent 

3. 



Very Poor 
5 
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2. Background /Or lent.at Ion? la it for adulta or 
children? Mlddle-claaa or lower-class? Cute or mature 
in reinf orcera? Environment appropriate ( joba^ 
familiea^ and dating va toya, fruity etc.)? Will 
developmental atudents relate to the overall context? 

3. Cognitive Ski 11a? Doea it aaaume more ability to 
manipulate concepta^ ideaa, intangiblea than moat 
developmental atudenta can handle? What apecif ic 
reaaoning/thinking capabilitiea and techniquee doea it 
aaaume? Doea it begin with what developmental atudenta 
would conaider "concrete"? 

4. Motivation /Attitude? Doea it aaaume curioaity, 
creativity^ a love o£ challenge, an intereat in 
learning p a belief in one' a ability to "make aenae" out 
of thinga (e.g. directiona, examplea)? How will aomeone 
with limited amounts of many of theae characteriatica, 
and perhapa expecting to become confused and defeated, 
react to the program? 

5. Ease of operationa? Can atudenta operate the 
program and aquipment eaaily? Is program very complex, 
in terms of what ia required to get into it and move 
around in it? Doea it require multiple key-atrokea, 
remembering aeveral types of commands without prompts, 
understanding computer terminology, etc.? Doea it 
require reading of complex manuals? 

B. EDUCATIONAL DESIGN 

Overall, doea the program repreaent a powerful learning 
experience for the atudent, in terma of underlying intent, 
deaign, etc.? (Note! the detaila of how the deaign ia 
implemented are in Part C, below). 

1 . Baaed on aolid teaching/learning theory? Doea it 
aeem to understand how developmental atudenta learn? Ia 
thia atated, or clearly implicit? What doea it aaoume 
about teaching/learning proceaaea? Ia there an explicit 
model or theory behind it? 

2 • Purpoaea and obiectivea clear? What doea it intend 
to do? Ia thia clearly apecified for the instructor? 
For the atudent? 

3. Content appropriate » accurate, biaa-free? What 
preciae knowledge, techniquea, concepta doea the program 
teach? Are theae conaiatant with our beat underatanding 
of what ia important^ and true? Are there factual 
errors? Doea it encourage atereotypic thinking, biaaea, 
aexiam, or antisocial behavior (e.g. violence), either 
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In the content or in the way the content is presented 
(graphics, relnforcers, etc.)? Remember that "content" 
Includes everything the student learns from a program, 
not just the disciplinary mater'.al Intended. 

4. Stimulates thinking an i creativity? Does the 
program force students to choose, ponder, think things 
through? Does It open up possibilities or foster "right 
answer" thinking? Does It allow multiple correct 
responses? Does It encourage "what-lf?" thinking? Does 
It show or Imply complex relationships among knowledge, 
or just Isolated facts? Does It spoon-feed, or offer a 
can-opener? 

5. Instructional sequence logical, modular? Does It 
flow from simple to complex, concrete to abstract? Does 
It Include all required knowledge and skills? Are the 
"bites" of learning the right size for developmental 
students? Is It slow enough, but not boring? 

C. INSTRUCTIONAL DELIVERY 

The factors below weigh not what was Intended, but how well the 
Intention was carried out. 

1. Keeps students active? What percentage of the time 
are the students doing something, vs time watching the 
screen? Are the activities more than picking "yes/no" 
answers? Do the activities force the students to 
wrestle with the material? Does It require so much 
wreatllng that they will become discouraged? Do they 
interact wlbh the computer In degrees and ways 
comparable to Interactions with a live Instructor In a 
one-to-one situation? 

2 . Handles wrong answers effectively? Does 1 t react to 
wrong answers with clues, hints, further Instruction? 
Does It just say "wrong"? Does It just give the right 
answer? Do the responses undercut the student ^s 
self-Image, confidence? Is the correct answer given 
after no more than 3-4 incorrect responses? 

3. Handles mechanical errors effectively? Does the 
program "crash" If students hit ESC or RET? How does It 
handle misspellings? Does It give cues for what to do 
when the student uses incorrect procedures, types badly, 
etc.? 

4. Input methods clear, simple, consistent? Is it hard 
for the student to remember how to give information or 
answers? Are complex codes or multiple key-strokes 
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used? Does it use the same method all the time? Do 
directions stay on the screen? Is it easy to Qsk for 
help in remembering? 

5. Adjusts to individual needs , strengths? Does it 
diagnose, and then adjust what is taught to what the 
individual needs? Can students skip ahead if they are 
competent in something? Doea it branch various ways re: 
level of difficulty , speed /pace , number of problems, 
etc.? Are tests frequent^ and appropriate? Do teats 
interfere with flow of learning, take too much time? 
Can students review previously-studied material without 
repeating? 

^- Responses build self -concept? Do the program's 
responses actively encourage the student? Do they 
maximize his/her motivation to continue? Do responses 
to both negative and positive answers do this? Are 
there a variety of negative and positive responses, or 
3ust one each? 

7. Students can move about freely in the program? Does 
the software use menus, maps, etc. to guide students? 

Do students control the pace o£ presentation of 
material, level of difficulty, number of problems, etc.? 
Can students stop, and then reenter later where they 
left off? Will a student see her/himself as in control 
of the machine? 

8 . Directions and examples clear, useful, frequent? Is 
everything spelled out? Is the language simple, free of 
jargon, and at the proper reading level? Are directions 
consistent, not subject to misinterpretation? Are all 
complex things (from a developmental student's point of 
view) both explained and exemplified before st.udents 
must respond? How easy is it to get confused? 

D. AFFECTIVE IMPACT 

In many ways, a computer's unique strengths address the 
affective more than the cognitive domain. 

1 . Graphics, sound enhance learning? Do they "speak 
for themselves", by substituting for expository text? 
Are they more appealing for right answers than for wrong 
answers? Are they overused? Are there screens without 
graphics? Can the sound be turned off? Do they 
distract or harmonize? Are they at the appropriate "age 
level"? Do they actually help the student understand 
(e.g. by graphically demonstrating concepts) ? Are they 
mainly for show and dazzle? What would be loat/gained 
if they weren't there? 
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2. Interesting and motivating? Will the student want 
to stick with the program? In drill and practice 
programs, will they want to return to it? Will it help 
them want to learn , or just to play? 

3. Screen designs effective? Does the layout help or 
hinder? Is it too crowded with text? Does it use 
delays, blinks, or reverse text to show emphasis? Does 
it keep information on after it is no longer needed? Is 
it ueed mainly as a scrolling textbook? Is it visually 
appealing? 

E. INSTRUCTOR USE 

The value of even a good piece of software diminishes to the 
degree that It increases the long-term burden on the instructor. 

1 • Minimum learning , preparation time? How long does 
it take the instructor to master the program? Is it 
easy to remember how to use it? How much preparation is 
required before using it with given students? 

2 • Instructor can adapt program content? Can the 
instructor add to, delete from, or change the problems, 
vocabulary, exerclsea, etc. to make them more pertinent 
for developmental students or specialized uses (e.g. 
technical vocabularies)? Can the instructor modify the 
program Itself? 

3 . Useable with varied student groupings, contexts? Is 
the program useful with individuals, small groups, 
nnd/or class-sized groups? Can it be used individually 
in a computer lab, or does it need an instructor 
available? Will Its use distract others in a learning 
lab? 

4 . Records student performance? Does it keep track of 
individual student work? What kind of data does it 
keep? How many students can it accommodate? How secure 
is the data from other students? Does it require 
students to have, and remember, secret codes? 

5 . Can serve several computers simultaneously? Some 
programs must remain in the disk drive in order to be 
used. Others can be removed, and used to start up any 
number of other computers. The former requires one copy 
per student, the latter requires one copy for all 
students. This can be a critical factor, given the cost 
consequences. If the disk cannot be copied, and each of 
several copies must be bought at full commercial price. 
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F. SUPPORT ELEMENTS 

"Software" in moat cases means more than just the material on 
the disk or tapc^ Itself. The other elements can often be 

Important In determining the quality of the package. 

1. Operating documentation helpful? Is it written In 
understandable English? Does It assume prior knowledge? 
Do students need to go through It before using the 
program? Is It accurate, complete, and written for the 
lay person? Does It use examples ? 

2. Supplemental materials appropriate? Are there 
supporting lesson plans, work sheets, activity Ideas, 
reference works, etc . ? Are they appropriate for 
developmental students? Are they Instructlonally 
useful? Do they mesh with the program? Does It seem 
like the program has been written as a gimmick to sell 
the supplemental :;naterlal? 

3. Works properly? Does It start up and run every 
time? Does It stop for no apparent reason? 

4. Uses available equipment? Doea It match what you 
have: type of computer, amount of memory, tape vs disk, 
printer, joystick or paddles, light pen, graphics pad, 
etc.? Does It need dual disk drives? 

5- Free from errors? Does It have spelling, grammar, 
calculation, etc. errors? Does It have programming 
bugs? 

Few practicing developmental educators are going to have the 
time to weigh all of these factors to the n'th degree. Nor Is 
that necessary. Rather, decide what Is the minimum you must 
have. In terms of some of the above characteristics. Design a 
form to concentrate only on those characteristics. Then simply 
do a quick scan of other factors, or use them In case a 
tie-breaker Is needed between two acceptable pieces of software. 
Your main tradeoff In evaluating software will be time versus 
money . If you have a lot of funds , then scrupulous evaluations 
may be less necessary. If you have a little money, then you 
will need to spend more time deciding on the few things you can 
buy. Be prepared also to decide that there's nothing out there 
that you care to purchase right now! 



SECTION VII. PREVIEWING 

A key assumption In this evaluation schema Is that you will be 
able to obtain copies of software for Inspection purposes. How 
do you do that? 
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Increasingly, the better-known companies will allow previewing: 
either with a formal ''on approval" understanding or with a 
thirty-day return policy. Many companies not printing such 
policies in their catalogs will allow you to return purchased 
materials if it clearly fails to meet your needs • They will be 
far more inclined to do this for educational institutions than 
for individuals, given the prevalence of individual pirating 
(illegal copying) of software. When in doubt, you should write 
or call in advance to get some definite agreement as to the 
conditions under which you can return software after it is 
purchased • 

Many of the beat prices on educational software can be obtained 
from discount houses. Unfortunately, most of these are the most 
atrict concerning previewing and returns; many do not allow it 
under any circumstances. 



SECTION VIII. PUBLISHED EVALUATIONS & RESOURCES 

Increasing numbers of software evaluations are being published, 
in magazines and through commercial or public groups. There are 
no common standards among them. Most, in particular, pay little 
attention to the intended audience, a major problem for 
developmental educators. Most provide both a narrative review 
and some type of rating scale, using such factors as 
performance, ease of use, documentation, error handling, etc. 
Often, the ^quantitative^ ratings will be middle range or better 
even if the narrative points to severe problems. 

Some evaluations are baaed on group judgements and formal 
processes; most are purely one person^ s opinion. They should be 
read with great caution, and when possible two or more reviews 
of the same software should be compared. Below are listed some 
recommended publications containing evaluations or descriptions 
of software pertinent to developmental education. 

Courseware Report Card /Secondary 
150 West Carob St. 
Compton, CA. 90220 
($60/5 issues) 

Educational Software Directory 
P.O. Box 263 
Littleton, CO. 80160 
<$22.50/Edition) 

EPIE Micro-Courseware PRO/FILES 
P.O. Box 620 
Stony Brook, N.Y. 11790 
($260/260 Reviews) 



EKLC 



C. NADE 1984 



Page 15 



Info world Magazine 
P.O. Box dSO 
Framingham, N.H. 01701 
($25/51 laauea) 

Journal of Courseware Reviews 

P.O. Box 2S426 

San Jose, OA. 93159 

<$7/Isaue) 

Microcomputers in Education 
QUEUE 

4 Chapel Hill Dr. 
Fairfield, CT. 52319 
($24/12 Issues) 

MicroSIFT Reviews 
NWREL 

300 S.W. Sixth Ave. 
Portland, OR. 97204 
(Price not set) 

Pipeline 
Conduit 
P.O. Box 0 
Oakdale, lA. 52319 
($15/3 Issues) 

School Microware Reviews 
P.O. Box 246 
Dresden, HE. 04342 
($25/2 Issues) 

There are mounds of articles on microcomputers, software, and 
the like. The following are six which are worth reading, as a 
sampler. 

Bork, Alfred, "University Learning Centers & 
Computer-Based Learning" , Journal of College Science 
Teaching , Nov., 1982. 

Caldwell, Robert M., "Guidelines for Developing Basic 
Skills Instructional Materials for Use with 
Microcomputer Technology", Educational Technology, Oct., 
1980. 

Emmett, Arielle, "Discovering A New Way to Learn," 
Personal Computing , Jan . , 1984 . 
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Myers, Darlene, "Why Micros?", Collegiate Microcomputer ^ 
Aug., 1983. 

"Quality Software: How to Know When You've Found Itl", 
Electronic Learning ^ Dec . , 1982 . 

Root, Jock, "The Schoolhouae Apple (interview with Steve 
Jobs)", Softalk, Jan., 1984. 



SECTION IX > NEXT STEPS 

This document ia a primer, and a beginning. A refined version 
will probably come out within the year, reflecting advances both 
in the field and in our knowledge of it. 

This ia also but the first in a aeries of documents on 
microcomputers to be produced by and for the National 
Association for Developmental Education. At least five other 
documents, along the lines of this one in format and length, are 
being considered, dealing with: 

o How to implement microcomputer learning in 
developmental education programs, 

o How to select microcomputer hardware for 
developmental education programs , 

o A compendium of effective software for developmental 
education . 

o Future trends in microcomputers as they affect 
developmental education programs, and 

o Instructional and teaching/learning theory issues in 
microcomputers for developmental education. 

Additionally, NADE's Computer Task Force is considering the 
utility and feasibility of creating formal user networks among 
developmental educators across the country . 

Suggestions or reactions to anything in this document are 
welcomed, and should be sent to: 

Curtis Miles 
Dean 

Piedmont Technical College 
P.O. Drawer 1467 
Greenwood , S . C . 29648 
(803) 223-8357 
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GUIDELINES FOR MAKING QUALITY PRESENTATIONS 
by Anna M. Kowalczyk 
Purdue University/Calumet 



Guidelines for Making Quality Presentations 

Making quality presentations often present a challenge to even those 
who enjoy public speaking because so much more is involved in delivering 
the message than knowledge of the subject • Attitude, method, and ability 
seem to make the difference in how successful the presentation becomes. 
Another important element in public communication is the message, which 
can be broken down into purpose, organization, language, and supporting 
materials. All of these elements, of course, interact and effect one 
another. Almost everyone can become an effective speaker by learning the 
principles of communication, and with practice the speaker can gain the 
needed self-confidence and ability. Although this sounds simple, actually 
It is very complex. 

After the subject has been chosen and narrowed to a theses statement, 
the following guidelines may prove helpful 

1. Speech content should be: 

reliable - content reflects dependable thoughts based on objective 
research and experience. 

vaTld - statements are true facts or acceptable opinions. 
rel event - content is applicable or pertinent to the central 
purpose or resulting conclusions. 

suitable - speaker uses content familiar to him; content relates to 
the occasion of the speech and to suitable audience response. 

2. Speech organization should be clear. All coordinate and sub- 
coordinate parts should be recognizable and tied together with 
transitions, which can be either direct or delayed. 

3. In speech delivery, careful use of all valid principles concerning 
method and technique should be employed, as well as effective use 
of voice and body. 

Since the confrence speaker is usually expected to draw up the 
agenda, introduce the topic and objective, keep the discussion on tract 
and moving, and summarize the conference results and the concensus, the 
speaker *s goal is aimed at achieving defined objectives. For this reason 
confrence speaking must utilize basic public speaking roles to accompolish 
the desired end. Therefore, a series of complex skills must be squired to 
provide the direct personal contact required to deliver the message. 2 

In the pursuit of effective presentation, modem communication guide- 
lines seem to Indicate that It Is more effective to concentrate on the 
proofs of one's own point of view rather than to present both sides of an 
issue. Aristotle in his study of communication defined rhetoric as 
discovering in the particular case what are the available means of per- 
suasion. ''^ He held credibility, the ability to shape a message in its 
most persuasive form for a given audience, as the speaker's most important 
attribute. He belived that rhetoric should express itself through the 
speaker's character, ability to evoke the desired emotional response in 
the audience, and the proof, or apparent proof, of the speaker's claim. 
All available evidence seems to support Aristotle's belief that persuasion 
is more likely to be achieved when people like, trust, and have confidence 
in the persuader. 
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How do you determine on whom you will rely? Most likely, the 
presence of certain qualities will make the possessor a high credibility 
source. Although there are differences in opinion on what these qualities 
should be, most analyses include character » personality , competence and 
intention. The way of effectively communicating the message, then, 
becomes a key to its acceptance as a quality presentation. The speaker, 
therefore, must decide whether his audience will more readily accept the 
argument by implication or by explicit message. 

Because the communication process Is considered to be most effective 
In a face-to-face setting, nonverbal stimuli and the opportunity for imm-— 
diate verbal feedback act as indicators of understanding or confusion, 
agreement or disagreement, thereby influencing both the speaker and the 
audience.^ The more sophisticated the audienced, the more under- 
standing, knowledgeable and articulate must be the speaker. Not only must 
the right words be selected but also the correct tone, order, and emphasis 
to convince and motivate the audience must be established. Of course, the 
more that is known about the audience, the more accurately their reactions 
can be measured. This is probably the most valuable communication skill a 
speaker can possess. However, since no two individuals see the world in 
exactly the same terms, the problem of interpretation becomes the sender *s 
concern. To achieve the greatest understanding and minimize ambiguity, 
especially with a large audience, the more simple, direct, and conven- 
tional should be the language and illustrations. The simple, direct, con- 
ventional approach, however, is not always the most effective means of 
transmitting the message. 

Fortunately, a variety of options is available for situations that 
clearly demand imaginative, indirect, and unconventional presentations. 
The use of visual aids, such as chalkboard diagrams and sketches, actual 
obj'^cts or replicas, posters, graphics, such as slides, summary charts, 
maps, cartoons, illustrations, etc., are options available to promote 
audience intrest.^ Also, the speaker* s image may be affected by the use 
of figures of speech metaphors, similies, parallels, analogies, and 
even obscenities. Caution is advised in the use of figures of speech 
because figurative language may confuse listeners. In particular, ob- 
scenity may offend even though that language may not be uncommon to the 
listeners. To avoid audience disapproval, careful audience analysis 
should be undertaken before obscenity is used. 

Other forms such as group discussions, questions and answer series, 
role playing, skits, poetry, and music often enliven audience enthusiasm 
and approval of the speaker. To instill the desired Intent in the 
listener's mind, the format employed often determines the effectiveness of 
the message and arouses audience intrest. However, it may not always be 
possible to win over an audience. Even though the message is delivered in 
rhetorical splendor, if the listeners' state of mind is resistant to the 
speaker's message, there is no way to compel the listeners' acceptance. 

In addition to the message form, the sender's verbal and non-verbal 
symbols can convey the desired meaning to the receiver. These inter- 
actions between speaker and listener can result in a complementary 
relationship if the verbal and non-verbal behaviors have been consciously 
calculated and transmitted through appropriate symbols. The reciever of 
the message, therefore, must be considered an active participant into whom 
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the meaning Is Instilled » and the sender* s task Is to send the message In 
the form which will not only elicit a favorable response from the receiver 
but will also motivate the receiver to pay attention. This desired res- 
ponse serves as feedback to the speaker and establishes a rapport between 
them. Although It Is perfectly possible to communicate without feedback, 
It Is a valuable gauge for the communicator to determine audience re- 
action. 

It Is obvious that effective delivery takes careful planning and 
practice. The following basic principles of delivery provide ways to 
improve your presentation skills:^ 

1. Make your Information relevant to the audience. Think 

about what you are saying. Speak Ideas rather than words. 

2m Strive to make your speech a dialogue. Use direct eye 
contact and a conversational approach. Get Involved with 
your audience. 

3. Analyze and adapt to your audience. Change your pace, 
volume, pitchy gestures, etc., as cues from the audience 
Indicate. 

Remember that the audience Is made up of people who need 
to feel that they are a part of the process. 

4. Use your body and your voice to reinforce your Ideas. Let 
your gestures flow naturally, thereby avoiding affectation. 

5. Make your message the focus of attention rather than calling 
attention to yourself. 

6. Exhibit enthusiasm and excitement about your subject. 

7. Practice frequently by rehearsing your speech. Use a tape 
recorder to hear, evaluate, and Improve your speech and Its 
delivery. 

Margaret Mead writes about still other dimensions of making effective 
presentations: 

The good confrence participant must... en joy multlsensory cues... He 
should enjoy attending to several levels at once, what Is being said, 
what Is not being said that one might expect to have been said, what 
other meanings are being carried by cadence and pace, what types of 
Images underlie an utterance, how an utterance parallels In some way 
a previous utterance, how tone of voice reinforces the manifest con- 
tent or negates It... 

The confrence style of discourse has to be attended to as If It were 
simultaneously exposition and poet, denotative and evocative, and 
such multiple attention must be sufficiently congenial so that the 
participants feel enhanced by participation, rather than diminished 
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and fatigued. 



Bducators will find themsolvos Increasingly Involved in group procoBs. 
Involving participants of different cultures and backgrounds, and with 
varying sensibilities. The participant must adapt and adjust to what la 
going on, must maintain a high degree of concentration, and must display 
such virtues as attentlveness, cooperation, and the ability to disagree on 
Issues without Involving personalities. The guidelines and suggestions 
presented here on the principles of effective communication, organization, 
rehersaly and accompollshlng those things that generate self-confidence 
should prove helpful In making quality presentations. The great equalizer 
Is experience, which should develop the positive emotional attitudes of a 
good speaker: enthusiasm, sincerity, and conviction. 
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NOTES 



^Kelly, Win, The Art of Public Address , Dubuque, Iowa, 
Wm. C. Brown Company Publishers, 1965, pp. 7 & 8 

^Koehler, Jerry W. and Slsco, John I., Public Communication 
In Business and the Professions , St. Paul, Minn., West 
Publishing Co., 1981, pp. 9 & 10. 

•^Aristotle Rhetoric 1, 1335, from Lane Cooper the Rhetoric of 
Aristotle , N.Y., Appleton-Century-Crof ts, 1932, pp. 7. 

^Koehler/Slsco, pp. 15. 

^Reld, Loren, Speaking Well, New York, McGraw-Hill Book 
Company, 1982, pp. 210-218. 

^Leth, Pamela C. and Leth, Steven A., Public Connnunlcatlon , 
Brooks/Vogel Series In Speech Communication, Menlo Park, 
Calif., Cummlngs Publishing Company, 1977, pp. 46. 

^Mead, Margaret and Byers, Paul, The Small Confrence ; An 
Innovation In Communication , Paris, Mouton & Co., 
1968, pp.19. 



EKLC 



27 



GUIDELINES ON RUNNING CONFERENCES 
by Valeriana Brown, Ph.D. 
Minneapolis Community College 
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Running a conference requires that several components be 
planned carefully. Three major conference components can be identified 

1) Program 

2) Location 

3) Logistics 

Everyone of these components has to be thouroughly planned. This 
means the planning has to address the smallest details, no matter 
how insignificant they might seem. 

Planning a conference is teata work, and every member of this team has 
to feel that his/her work is crucial, because it is. The planning 
process for each of the above components has to identify tasks, set 
a timeline to accompolish the tasks, and assign the tasks to team 
members. All of this done in an ordely fashion will gaurantee a 
successful conference. 

Let us now address each of the components. 
1. Program 

The organization of the conference program is crucial. Usually the 
program is organized around a topic ot theme. What are then the major 
factors in choosing a conference topic? 

— Timeliness and relevence to what is happening in the field seem 
to be very important, since it will be the "eye-catcher" for par- 
ticipation. 

—Audience - determine what oudience(s) the conference will attract 
and accordingly develop the topic. If the conference is for prac- 
titioners in the field have sessions of revelence to them, if it is 
for administrators, have sessions that will appeal to them. If the 
audience is varied, then include a variety of sessions. A note of 
caution - if you cannot attract a significant number in each group, 
then you might want to address only one group (this is important for 
local conferences or workshops). 

— Geographical location - what might be a "hot" topic in Texas, might 
not serve the needs of Ohio. 

The selection of a topic might require a fromal or informal needs 
assessment, whether it is a local or national conference you are 
planning. 

Once a topic is chosen, then the program can be developed from the 
"naturals" of the topic or a call for papers if this is a conference 
for 300 or more people. Either way spearkers who can address the topic 
have to be selected. The selection of speakers is usually done by 
speaker reputation, word of mouth, experts in the field and sometimes 
unknown speakers who seem to have a good presentation. However, some 
reputable speakers will help your conference. 
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2. Location 



The choice of location has to be easily accessable to as many potential 
participants as possible. 

For a large conference consider the cost of air travel from different 
departure points, accessibility of nearest airport, parking arrangements 
popularity of the site, adequate facilities, and cost of hotel and food. 
When selecting a hotel Inquire about: 

- experience of hotel staff In running a conference. 

- previous clients (check with them about quality of service) 

- experience In handling a group of your group* s size 

- discount on room rates (at least 20%) 

- complimentary room (usually for each 50 rooms one complementary room) 

- free meeting space If you hold at least one meal event 

- advance registration service 

- room size/ set up 

- free parking 

- accessibility to food and entertainment 
-> limousine service to and from airport 

- weekend rates for conference participants 

3. Logistics 

The logistics of a conference such as meeting rooms, equipment (AV and 
other), schedule of conference, pre-conf erence workshop, tours, entertain- 
ment, selection of menus, arrival and departure times, transportation to 
and from airport, conference publicity are very Important to the smooth 
running of a conference. Logistics of a conference Involve many details. 
It Is easier when several members of the conference planning team take the 
responsibility for some of the logistics of a conference are going to 
enhance or hinder your program. It Is CRUCIAL that they are carefully 
planned and followed through. 

Once the program and location are decided the following arrangements 
need to be made. 

- Publicity - publicizing the conference Is obviously of exterme 
Importance. Brochures are the most common publicity Item. A brochure 

Is going to let the potential attendees know about your conference. Make 
It concise but complete. This means location, dates, cost of registration 
and hotel, program, other events, contact person(s) should be Included. 

How to publicize your brochure? 
For local conferences - mall to local colleges, local newspapers, local 
news-letters, local radio and personal contact with local members. For 
national conferences - association newsletters, journal advertisements, 
bulk mailing, PR at related conferences, mailings to college presidents 
and deans . 

- Meeting rooms — your program will dictate how many and what size 
rooms you will need, (work closely with hotel staff) 

- Equipment — ask your speaker what they need (overhead projectors, 
slide projectors, flip charts, markers, computers, etc.) Hotels usually 
rent or provide equipment. 
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- Tours and educational visits — visits to neighboring college programs 
hletoricul tours some event emphasizing local culture. Contact the college 
staff for necessary arrangements to visit. For Historical tours and cultural 
events contact the Chamber of Commerce. 

- Entertainment and Hospitality Suite — These are extras that promote a 
good coaf*»i*ence climate. They allow for interaction among participants, ex- 
change o:: ideas, and a cohesion among members. Some ideas on entertainment — 
have fc'ik singers or a dance as part of the program; have participants organize 
an entertainment evening; or arrange for a dinner/show event. 

- Ground transportation — inquire if hotel provides this service and let 
participants know ahead of time. 

- Meals — the selection of menus does not need to be ostentatious, but 
varied and appealing (hotel staff are usually helpful). If you are planning 
a local one day workshop — use a box lunch — it saves time. Coffee breaks 
also need to be included. 

- Cost — This is an important item. If you do not have any subsidy to 
run the conference, workshop, the registration fees will have to cover speaker 
expences, brochures, mailings, coffee breaks, and any other meals that are 
included in the program. The cost is divided into two catagories, fixed costs 
and changable costs. The speaker, brochure, mailings, are fixed costs because 
whether there are 20 participants or 200, this expenditure does not change. 
Meal costs vary depending on the number of participants. This determines how 
many participants are needed to break even or make a profit whichever is the 
goal of the conference. 

Sometimes local companies or industries will sponsor an event or coffee break 
in exchange for an exhibition space or publicity of their brochure. 

- Name tags and folders — Have name tags and folders ready for each partici- 
pant to pick-up at registration. Folders - should include an updated conference 
program, information on resteraunts, entertainment, cultural events and historical 
background on the area where conference is held. Sometimes publishing companies 
will provide brochures with information on recent publications. It is helpful 

to include blank sheets of paper and a pen. 

- Exhibition — exhibits of materials help give the conference or workshop 
the feeling of "in the now". Exhibitors usually will sponsor events or provide 
materials to participants. 

A check list is provided for easier planning of logistical details, also a 
sample evaluation form is included, which can be modified depending on the nature 
of the conference. 
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CHECKLIST FOR MEETING AND 
CONFRENCE PLANNING AND ARRANGEMENTS 



Individual 

Responsible Item Notes of Comments 

FACILITIES 

Hotel Contact person 

Location Date(s) Time 

Speakers 

Name, address, and phone number list 

_^ Initial contact 

Confirming received 

Lodging confirmed 

Fees and travel expenses confirmed 

Fees and travel expenses paid 

A\idio-visTial needs determined 

Biographies requested (photo?) 

Other special arrangements 

Meeting Rooms 

a. Main meeting room 

Optional rooms 

Cost 

Lighting 

Equipment 

Size 

b. Small meeting rooms 

Optional rooms 

Location 
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Facilities (Continued) 
Cost 



Numbe r 

Table & chairs 

c. Exhibit area 

Guest Rooms 

Complimentary rooms 

Number of each quoted figure 

Flat rate 

Any special rate 

Guarantees needed 

Checkout time 

Payment allowed (credit card 
Food Service 

In hotel 

Adjacent 

Menus 

Guarantees 

Meeting Service 

Ice water 

Coffee 

Security (lockup, etc*) 

Contact person 

Lighting control 

Heat control 
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Facilities (Continued) 

Registration Facilities 

Proximity to main entrance 

Equipment available 

Transportation 

Courtesy service to airport 

Parking area 

Special Equipment 

Audio-visual materials available 

Telelecture potential 

Computers 

Traffic Flow (Foot) 

Access to main meeting room _ 

Access to small rooms 

Location of rest rooms 

Audio -Visual Needs 

PA system 

Lectern(s) 

Screen(s) 

Chalkboard, chalk and erasers 

Chartboards 

Overhead projector(s) 

Slide projector(s) ^ 

Movie projector(s) 

TV viewer(s) 

TV tape recorder(s) 

Flannelboard(s) 

Extension cords (3 wire) 
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Audio-Visual Needs (Continued) 

Felt pens (variety of colors) 

Transparency markers (variety of colors) 

Tables - for exhibits 

Tables - for setting up presentation 

Tables or stands for AV equipment 

Paper and pencils 

Spare parts (each piece of AV equipment) 

REGISTRATION 
Pre-registration 

Solicitation of participants 

List of participants ^ 

Packet for mailing 

a. Hotel registration forms 

b. Room selection and rates 

c. Address and phone 

d. Location map and instructions 

e. Registration fee 

f . Local intrest items 

g. Time and place of registration 

h. Registration deadline ^ 

Fee receipts 

Name tags , 

Meal tickets 

Confrence programs 
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Pre-registration (Continued) 

Register those not pre-registered 

Extra pre-registration packets 

Facilities map 

Special events program ^^^^ 

Clerical help 

Greeter - host 

Chamber of Commerce packet 

Participant workshop folder 

Cancelations 

Late arrivals 

PERSONNEL 

Registration clerks 

Registration greeter - host 

Audio-Visual attendent 

Meeting room help (lights, heat, clean-up) 

CONFRENCE OPERATION 

Opening 

Greetings 

Introduction of speaker(s) 

Operation of TV viewer 

Menus (Lunch and Dinner) 

POST CONFRENCE 

Departure schedules 

Transportation to airport 

Confrence evaluations 

Mailing lables for all participants 

Mailing materials to participants 

Er|c 36 



Poet Confrence (Continued) 

Retrieval of lost property 

Summarize and pay all expenses 

Return rented equipment _ 

Return borrowed equipment 
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PROGRAM EVALUATION: A PRIMER 
Darrel Clowes & Belinda Anderson 
Virginia Tech 



AN EVALUATION PRIMER 



Darrel Clowes with Belinda Anderson 

This short pi^jct: tj intended to provide people working in develop- 
mental education a brief introduction tu the complex topic of evaluation, 
some "first principles" behind the paradigm set forth for evaluation of 
developmental programs, and guidance in the first steps necessary to set up 
the evaluation procedure. Evaluation of educational programs once was seen 
as an application of the scientific paradigm where the desired ends were 
known, measurable goals were established for the activity, and evaluation 
consisted of measuring goal achievement against goals established. This 
model^ assumed that goals were known and agreed upon by all the members of 
the institution and that the goals set v/ere subject to accurate measure- 
ment. Gradually evaluation workers became convinced that agreement on 
goals was a rare phenomena within an institution and :hat establishing 
precise measures for unclear goals was disfunctional . Alternative evalu- 
ation paradigms were developed that did not assume concensus on goals but 
rather al lowed the evaluation process to modify declared goals or to 
provide insight into the actual goals achieved by the educational activity 
(see Clowes, 1981 for a fuller discussion of this literature). This primer 
will arbitrarily adopt one evaluation paradigm and set out its underlying 
assumptions and limitations; the reader must be aware that many alternative 
models exist. 



FIRST PRINCIPLES 

The "first principles" behind the evaluation paradigm for develop- 
mental programs set forward here illustrate the assumptions underlying the 
evaluation described. The paradigm and its assumptions represent conserva- 
tive programatic goals and evaluation strategies that should provide a 
reasonable base for evaluation of any developmental activity and for the 
use of the evaluation results. The first assumption is that the function 
of the developmental activity is to remediate students' academic defi- 
ciencies so that students may function successfully in the mainstream 
curriculum courses of the institution. Academic concerns for adequate 
basic skills development and protection of the quality of the mainstream 
curriculum are accepted as the paramount i.istitutional concerns and the 
dominant concerns of the developmental activity. Concern for the psycho- 
logical and social development of the individual student and their inte- 
gration into the institution are not the primary focus of the developmental 
effort (for further discussion of this distinction in programatic thrust 
see Cross, 1976; Roueche and Roueche, 1977; Roueche and Snow, 1977; and 
Clowes, 1979). 

A second principle or assumption follows from the first. If the 
primary goal of the developmental activity is accepted as preparation for 
success in the mainstream curriculum, then the evaluation of the develop- 
mental effort must occur in two parts. First, the support lab activities, 
the developmental course work, or the work in a self-contained develop- 
mental program must be seen as a single level of activities that occur 
conceptually prior to entrance into the mainstream courses. Evaluation of 
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these activities is useful, but utility comes in terms of formative eval- 
uation designed to improve the level of academic skill development within 
the specific support lab activity, the course work, or the total program. 
Evaluation of this segment of the developmental activity produces informa- 
tion useful for the improvement of specific educational activities. 
Second, assessment of the effect or success of the developmental activity 
must be made in terms of the success students experience in making the 
transition from the developmental program to the mainstream curriculum, 
from specific developmental (basic skills) courses to the related advanced 
basic skills courses of the mainstream curriculum, or from the support-lab 
based activities to the related mainstream advanced basic skills courses. 
Given the paradigm used here, effectiveness for the developmental program 
means student success in making the transition from developmental activ- 
ities to successful completion of the related mainstream courses and 
ultimately success in completing their program and graduating from the 
institution. This second principle of evaluation in stages is critical. 
Separating evaluation of what occurs within the developmental activity from 
evaluation of what occurs after the transition from the developmental 
activity to the mainstream curriculum allows a focus upon each stage. 
Evaluation of the first stages provides information useful for improving 
the instructional design of the developmental activity but not useful for 
assessing the effectiveness of the developmental effort. Evaluation of the 
next stage - transition to the mainstream curriculum - provides information 
useful _ for assessing the effectiveness of the developmental program in 
preparing students to survive in the mainstream of the institution but not 
particularly useful for determining changes in the instructional program. 

A third principle is to use quantitative and readily available data. 
Quantitative data is numeric data represented by test scores, course 
grades, persistence rates and various other measures of academic achieve- 
ment and persistence. Quantitative measures are used because they are easy 
to work with, communicate clearly to many people, and have a face validity 
that gives credence to early evaluation efforts. When the evaluation 
process matures, qualitative data can be introduced to allow new forms of 
Input, to cope with areas difficult to measure, to allow the introduction 
of subjective data, and to assist in developing insight into operative 
goals of a program. The quantitative data used in this model exist in the 
Institution and can be easily obtained. Demographic data on students are 
contained in the student record files; achievement measures exist in 
student records, in departmental records or testing offices, and measures 
of persistence at the institution exist in the student transcript file 
maintained in the registrars' office. The use of readily available quanti- 
tative data allows the initial evaluation effort an opportunity to make its 
first contributions without imposing serious data collection demands upon 
the institution. Using existent data also allows time in which to estab- 
lish the real needs for additional data and to plan the opportunities and' 
instruments needed to obtain it. 

A fourth principle is to assume multiple audiences for the evaluation 
data generated. Faculty and staff working in the developmental activity 
will be particularly interested in evaluation of the activities internal to 
the developmental effort, faculty and administration involved with the 
mainstream courses will be particularly interested in the second stage of 
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transition into the mainstream courses and curriculum, and administrators 
will be particularly interested in the cost-effectiveness data. This 
paradigm attempts to provide basic evaluation data of interest to each of 
the primary constituencies within an academic institution. 

The final principle is to take a modest stance and not attempt to 
relate the evaluation effort to the achievement of goals that are not 
easily agreed upon or goals whose achievement is difficult to measure. 
Modest goals such as completing developmental classes with a grade of "C" 
or better or attending a learning support center for a specified number of 
sessions or over a specified time are goals easily accepted and understood 
for the assessment of a developmental activity. Persistence in the next 
related mainstream course and completion of that course with a "C" or 
better is a readily accepted measure of the transition stage. Completion 
of a specified number of credits after the developmental activity or 
persistence to graduation are also easily accepted goals. Less acceptable 
goals are abstractions like improving motivation, self-image or concept, or 
assisting the student to make appropriate career and life choices. The 
developmental goals are no less desirable as program goalsj however, they 
are far less readily acceptable or measurable. They are not appropriate 
goals for the early efforts at evaluation. They have a place, but that 
place is much later in the development of a full evaluation process (for a 
further elaboration see Clowes, 1981, 1984). 



FIRST STEPS 

Like Caesar's Gaul, this evaluation paradigm is divided into three 
parts: evaluation of the developmental activity itself; evaluation of the 
transition to the mainstream curriculum; and evaluation of the cost- 
effectiveness of the developmental activity. 

EVALUATION OF THE DEVELOPMENTAL ACTIVITY 

The developmental activity must first be stipulated to make clear the 
parameters of the activity to be evaluated and to inform estimates of 
reasonable success. Catagorization is helpful. Is the developmental 
activity a series of support labs and/or tutorial activities operating to 
support concurrent mainstream courses? If so, the activity has a "loose" 
relation to the academic mainstream of the institution. If the activity 
consists of a series of separate courses available for the student but 
offered by the various academic departments, then th? activity has a 
"close" relationship to the academic mainstream. ,he program is 

a separate body of courses and labs with a separate faculty and 
requirements for completion before the student enters the mainstream 
courses, the activity has a "tight" relationship to the mainstream. 
Control over the developmental activities and accountability for program 
success vary directly with "tightness" in this model. A "loose" program 
has less opportunity to impact the student in the program and more 
competing pressures influencing the student than does a "tight" program. 
Defining the students in the developmental activity is also affected by 
this categorization. In a "loose" program developmental students are those 
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participating in any support lab or tutoring activity. This is a weak 
definition since students will have degrees of program involvement with 
enormous variation; this is reflective of the weak relationship between the 
program, the student, and the mainstream academic programs. Evaluation is 
especially- unreliable in this situation because of the tenuous relation- 
ships among the activities, the students, and the curriculum. The 
"tighter" the activity, the clearer the definition of the developmental 
student and the relationships within the curriculum become. 

Measures must next be identified first to describe the developmental 
students and compare them to the general population and second to assess 
achievement within the developmental activity. A program ought to know who 
it is serving and if this client group is representative of the total 
population at the institution, a special sub-group, or a unique and other- 
wise unserved group. Information of this sort has profound bearing upon an 
evaluation of program goals. Routine data can be gathered from the 
registrar's office on standard demographic variables like age, race, sex 
and possibly enrollment status (part-time/full -time) and high school 
curriculum (vocational, general, academic). Data on testing programs for 
basic skills can be gathered from departmental or college level offices. A 
comparison for goodness of fit between developmental students and all 
students at the institution yields useful information. Achievement within 
the developmental activity can be assessed using three existent measures: 
grades in developmental courses, persistence to completion in developmenta 
courses and in mainstream courses (if taken), and a calculation each tern 
of hours attempted/hours earned. There three measures will provide basic 
information on the progress of students within the developmental program. 
Additional measures can easily be added. Identifying students by identi- 
fied areas of basic skill deficiency can be helpful. Math, reading, 
writing, and study skills are traditional basic skill areas. Tracking 
students with only a math deficiency or only a reading deficiency may 
reveal differential rates of success in the program and suggest program 
improvements. Alternatively, combinations of deficiencies might be looked 
at to determine rates of success. Do students with writing deficiencies 
usually also have reading problems? Do students with multiple deficiencies 
(math, reading, and writing) have a rate of success different from those 
with only a single basic skills deficiency? These questions can easily be 
asked and the information used to reassess the instructional program within 
the developmental effort. Parenthetically, pre- and post-test scores are 
not recommended as measures at this point. Gain scores over a ten to 
fifteen week term are questionable at best, and the relationship between 
gain scores and the ability to succeed in the academic mainstream is 
inconclusive and varies by institution (Richardson, Fisk and Okun, 1983). 

EVALUATION OF THE TRANSITION TO THE ACADEMIC MAINSTREAM 

The transition to the academic mainstream varies with the develop- 
mental activity category. A "loose" activity will have the student taking 
mainstream courses concurrent with the developmental activity. In this 
situation the transition occurs within the same term as the developmental 
activity. The transition must be assessed by identifying developmental 
students, identifying the mainstream courses in their area(o) of basic 
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skill deficiency in which they have enrolled, and tracking their achieve- 
ment in that course or courses as measured by course completion and course 
grade. A measure of hours attempted/hours passed for the term will give a 
more general measure of success in the transition. Assessment of the 
transition is cleaner with the "close" or "tight" developmental activity. 
The same basic evaluation design is followed, but the dividing line between 
the_ developmental, activity and the mainstream is clearer. It is also 
easier to refine the assessment by again identifying areas of deficiency 
and following student success by area or as a function of combinations of 
basic skill deficiencies. These measures will provide evaluations of the 
developmental activity one, two, three or more terms after a student is 
admitted to the institution and enrolled in developmental activities. 
Further data can be gathered by accumulating grade point averages and rates 
of graduation or program completion. These data are useful for determining 
the effectiveness of the developmental activity in moving students assigned 
to developmental work into the mainstream curriculum and through the 
institution. 

A caution, when an evaluation program as set forth here is in place 
and the data described here is gathered, the real task is only begun. With 
both formative information on the developmental activity and summative 
information on the transition phase, the serious work can begin. Once you 
know the level of success of a program given the goals set in this 
paradigm, the real question is "are those goals appropriate for this 
program and this institution?" Now the true purpose of program evaluation 
comes to the fore as a reassessment of curriculum goals and as a starting 
place for the curriculum development process. A further reading in the 
evaluation literature (see bibliography) will show alternative evaluation 
models and measures which might be introduced to improve or replace the 
paradigm set forth here. 



EVALUATION OF COST-EFFECTIVENESS 

Cost-effectiveness here is used in its simplest sense to assess 
whether the developmental activity is self-supporting. Institutions with 
good management information systems would already have this information, so 
the intent here is to provide a reasonable approach for gathering and 
expressing cost effectiveness data for institutions without existent 
management information systems. The first step again is to categorize and 
define both program and students on the "loose," "close," and "tight" 
program pattern described earlier. A "loose" program would have little 
opportunity to produce revenue since student credit hours are not often 
generated through support lab or tutorial activities and fees are not 
usual ly_ charged. For some "loose" programs and all other developmental 
activities, the data collection and display described should yield useful 
and useable_ results. Once student credit hour enrollment in developmental 
activities is established, a calculation of gross revenue generated can be 
made. Credit hours generated in developmental activities can be projected 
into revenue earned from tuition charged students, specific fees associated 
with_ developmental activities, and state and local support. Special 
funding support through grants and other support from external sources can 
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than be added. Together these monies represent the gross revenue of the 
developmental activity. Direct and indirect costs are then determined and 
subtracted from gross revenue to determine the net revenue generated (if 
any) by the developmental activity. Direct costs are instructional costs. 
The usual method is to determine the number of sections taught by full-time 
faculty and the number taught by part-time faculty in an academic year. 
The cost per full-time faculty section is the average faculty salary (not 
including fringe benefits which are covered in indirect costs) divided by 
the average number of sections taught by a full-time faculty member in an 
academic year. The average cost of a part-time section can be obtained 
from the department head or dean. The same source can supply an actual or 
estimated cost for instructional supplies for the developmental activity 
for the academic year. Combined these data represent the direct instruc- 
tional costs of the developmental activity. Indirect costs are the 
apportioned costs of operating the total institution. General estimates of 
indirect cost have been developed over the years; a range is presented 
here. A conservative estimate of indirect costs is 80% of direct instruc- 
tional costs for a program; tfiis estimate would provide the most positive 
assessment of cost effectiveness. An average estimate of indirect costs in 
100% of direct costs and a high estimate would be 110%. 

Cost effectiveness in this model is determined in a conservative 
manner by calculating gross revenue generated by the developmental 
activity, subtracting calculated direct costs and estimated indirect costs 
of the activity, and expressing the remainder as the net revenue generated 
or the cost incurred by the developmental program. This calculation takes 
no note of the revenue generated by students who successfully make the 
transition into the curricular mainstream. From the evaluation information 
generated earlier a reasonable case can be made that some students would 
not have persisted in the institution without assistance from the develop- 
mental activity. Thus an informed estimate of the proportion of transi- 
tioning students' credit hours attributable to the successful working of 
the developmental activity can be made. A conservative estimate of 5-10% 
of the average credit hours generated by students who successfully complete 
the^ transition would seem reasonable and politically defensible as an 
addition to gross revenue. Most programs will show significant net revenue 
generated without the addition of an estimate of successful transition 
student's contribution, but that addition is availab" '? desired. 



CONCLUSION 

This primer is intended to provide guidance in establishing an evalua- 
tion program. It is designed as a general model useful for most institu- 
tions and taking the most conservative positions on program goals and the 
measures used to assess those goals. This model or paradigm has two 
distinctions. First, it separates the evaluation effort into three phases: 
a formative evaluation phase that focuses upon the developmental activity 
itself; a summative phase that' focuses upon the transition into the main- 
stream curriculum; and a cost effectiveness phase. Second, this model 
posts arbitrary goals for the developmental activity. These goals are 
associated with a remediation emphasis and an emphasis upon academic 
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quality concerns at the institution. While these goals are generally 
present in all institutions, the priority and combination of goals may not 
be appropriate. An assumption behind this paradigm is that an institution 
will gather data as set forth here but will then use that information to 
redesign the goals and or the evaluation activity to better accommodate its 
own unique reality. 
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A major benefit of this book is its emphasis on evaluating 
student services agencies and thus affective influences in higher 
education, this book also emphasizes the relationship of political 
issues to evaluation. The section on evaluating the learning environ- 
ment is especially helpful. 

Guba, E. G., and Lincoln, Y. S. (1981). Effective Evaluation; Improving 
the Usefulness of Evaluation Results Through Responsive and Natur al fs-" 
tic Approaches . SanFrancTscol Jossey-Bass. 

This is an argument for the use of naturalistic inquiry methods 
where they are . appropriate. The review of evaluation as a field is 
exceptional and prepares the base for supporting responsive evaluation 
as the approach of choice. The arguments are lucid and often elegant, 
the distinctions helpful, and the writing clear. Here one finds 
excellent minds engaged in a fascinating and yet practical probing of 
the purpose, limits, and proprieties of evaluation. 
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Jossey-Bass. 
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IS notable for its practical approach and its emphasis upon the 
political purposes of evaluation. 

Newman, J. E. , and Hinrichs, J. R. (1980). Performance Evaluation for 
Professiona l Personnel . Scarsdale, N.Y.: Work in America Institute. 

Reviewing current literature, the author defines and describes 
the purposes of performance evaluation, addresses the issue of 
evaluating professionals, describes facets of performance evaluation, 
and discusses the feedback process. The second half of the book 
contains abstracts of articles written on performance evaluation. 

Patton, M. Q. (1980). Qualitative Evaluation Met hods. Beverly Hills, 
Calif.: Sage. " 

Patton provides an excellent guide to both when and how to use 
qualitative methods. The sections on evaluation design and on the 
nature of qualitative data are especially helpful. This book then 
carries through the implementation stage with sections on collecting 
and analyzing qualitative data. A real strength is the liaht writing 
style and brilliant use of parables as chapter beginnings. 

Popham, J. W. (1975). Educational Evaluation . Englewood Cliffs, N J • 
Prentice Hall. 

This book traces the historical development of educational 
evaluation and describes major evaluation concepts and terms. The 
author categorizes evaluation designs using a very understandable 
system. Chapters on criterion-referenced measurement and affective 
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measurement are quite helpful.* Popham's easy style of writing makes 
this book a delight to read. 

Smith, H. P., and Brouwer, P. J. ('077). Performance Appraisal and Human 
Development . Reading, Mass.: Addison-Wesley. 

Focusing on utilizing human talent effectively, this book dis- 
cusses the psychological aspects of performance appraisal . It 
provides information on the principles of human growth and development 
and how these principles relate to performance appraisal. Sample 
performance appraisal interviews are included to demonstrate effective 
techniques for managers. 

Sullivan, L. L. (1979). Sullivan's Guide to Learnin.-^ Centers in Higher 
Education . Portsmouth, N.H. : Entelek. ' 

This is a report of a survey of learning assistance centers in 
postsecondary institutions in America and Canada. Its strengths are a 
good response rate (about 50 percent) and volumes of data arranged in 
every conceivable way. This study updates Devirian's 1974 study and 
provides baseline data. Its weaknesses are a lack of definition of 
learning assistance centers, which clouds the findings; poor descrip- 
tions of the methodology so independent interpretations are 
impossifele, and lack of extensive interpretation of the data. 

Trillin, A, S, and Associates. (1980). Teaching Basic Skills in College . 
San Ncisco: Jossey-Bass. « 

This book contains a chapter on evaluation very helpful to 
experienced practitioners working with an established program. A 
number of evaluation options are addressed and critiqued. Design and 
measurement issues are raised and related to various evaluation 
options. 

Walvekar, C. C. (Ed.) (1981). New Directions for College Learning Assis- 
tance : Assessment of College Learning Assistance Services , No. ^ 
San Francisco: Jossey-Bass. 

This edited collection ranges from "how to" chapters to theoreti- 
cal chapters on the evaluation of learning centers. Although not 
systematic, it is a good source of ideas and examples. It is one of 
the few works addressed specifically to this topic. 



NOTE: Several of these annotations are drawn from pages 113-115 of the 
Walvekar volume. 
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Critical Issues and a Research Agenda for 
Developmental Education 
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Developmental education faces troubled times. Where 
the seventies saw an emphasis on increased access to higher 
education and active federal and state support of post- 
secondary developmental education , the eighties have brought 
an emphasis on academic quality at the expense of access and 
curtailed support of developmental education. Throughout 
the seventies pleas for meaningful research and evaluation 
went unheededr but now the demands for both are enormous. 
Research reports are emerging; they range from positive 
(Boylanr 1983) to mixed (Kulik, Kulik & Shwalb^ 1983) to 
negative (Richardsonr Pisk & Okun, 1983) assessments of 
post-secondary developmental education. The Executive 

Committee of NADE has moved to address this situation on 
several fronts. As one part of * that effort the NADE 
Research and Evaluation Committee was asked to address 
information needs of developmental programs by producing An 
Evaluation Primer (Clowes & Anderson^ 1984) and by 
developing a "research agenda of critical issues in 
developmental education." This paper reports the results of 
a questionnaire survey administered in Winter 1984 and its 
use in identifying critical issues and a research agenda. 



The purpose of this study was to identify the critical 
issues facing developmental educators and from the 
identified issues to develop a research agenda. The 
population of developmental educators chosen for the study 
was the membership of NADE. The sample was the membership 
of the Research and Evaluation Committee and the Executive 
Committee as two groups representative of the membership but 
particularly attuned to issues and research concerns. 
Therefore this study was designed to identify the critical 
issues for developmental education as perceived by 
professionals active in the field. The author then used the 
identified critical issues to prepare a research agenda. 
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The population of interest was developmental educators 
as represented by the membership of NADE. The specific 
sample used was a combination of the 32 member Research and 
Evaluation Committee (R & E) and the 6 member Executive 
Committee of NADE. The R&E membership has one 

representative from each state affiliate plus a core of 
voluntary members. These individuals are geographically and 
professional representative of NADE membership (see Table 
1) . They also have some claim to special interest or 
knowledge in research concerns • The Executive Committee is 
elected from the general membership and therefore should 
"represent" that group; they also have special experience 
helpful in identifying critical issues because of their 
roles with NADE. Thus the total sample was selected to be 
representative of NADE but with special expertise in the 
area of interest. 

While the assertion that the sample selected is 
representative of the NADE membership must be accepted at 
face valuer Table 1 shows that the respondents are similiar 
to the non-respondents in the sample. The Chi-square test 
showed no statistically significant differences between 
respondents and non-respondents among the R&E membership. 
Twenty of the 32 R&E members returned usable surveys for a 
62.5% response rater 6 of 6 Executive Committee members 
respondedr and the combined response rate was 26 of 38 or 
81% . 
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N = 32 



TABLE 1 

Comparison of Respondents and Non- 
Respondents of Research and Evaluation 
Committee. (Row Percentage in Brackets) 



Respondent 
Non-Respondent 



Four-Year College 
Personnel 



Faculty 
Only 



7 (64) 
4 (36) 

11 (1 1) 



Faculty 
Administrator 



6 (67) 
3 (33) 

9 (TOO) 



Two-Year College 
Personnel 

Faculty Faculty 
Only Administrator 



3 (50) 
3 (50) 



4 (67) 
2 (33) 



6. (100) 6 (100) 



X' shows no significant differences. 
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Surirsx InsixiiioenJ: 



The survey instrument used (Appendix A) wns developed 
by the author. It was constructed after a review of the 
monograph and journal literature and of the ERIC data base. 
Three general areas of critical questions for developmental 
education were identified: those internal to the program, 
those external to the program, and justifications for 
developmental education. A fourth general area on criteria 
for program evaluation was added to contribute to the 
Evaluation Pioneer project mentioned earlier. There were 
six items under each general question except for the program 
justification question which had eight. 

The assumption of validity was supported by all 
respondents' ability to complete the instrument and the very 
modest number and rating of items added. Reliability is 
supported by comparison of responses by the R&E membership 
and those of the Executive Committee. Responses were scored 
1 for "highest priority" to 5 for "lowest priority"; means 
were calculated for each general question and for each item 
and then rankings were assigned. For these two groups the 
ranking of general questions was similar (see Table 2) while 
the ranking of specific items was very close. The two 
groups each had eight of their ten highest ranked items in 
common. . A third group was used as a check on reliability. 
Seventeen graduate students in community college/higher 
education programs from the author's classes also completed 
the survey. Although only two of this group had experience 
with developmental education, seven items from their top ten 
ranked items were also in the top ten of each of the other 
groups. Finally, correlation coefficients were calculated 
among the items under each general question. Three general 
questions showed strong positive correlations on 3 or fewer 
of the twenty-one possible pairings; the question on 
justifications of developmental programs, however, had one- 
third of its possible pairings with positive correlations. 
This suggests that the items are generally independent of 
each other except under the justification question. The 
items ranked as "above average" priority or higher were 
independent of positive correlations in all but one 
instance. Thus an assumption of reliability is supported by 
a pattern of similiar responses with three different groups 
and a test for between item correlation that found few and 
generally unimportant positive correlations. 

Each item was scored on a 5 point scale by assigning 1 
point for "highest priority" to 5 points for "lowest 
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priority." All calculations were done for the R&E 
respondents, the Executive Committeef and for a combined 
sample. Means were calculated for each item and then a mean 
of all items under each general question was calculated as a 
mean for the general question. Item means were converted to 
a priority ranking; all items with a mean of 2.0 (above 
average priority) or below were accepted as high priority 
items. General question means were calculated and the 
general questions were ranked; individual item means were 
also calculated and the items receiving a 2.0 or leas were 
ranked. 



Findings are presented in three stages. First, means 
and rankings of the general questions are presented. 
Secondf the ranked items are presented in relationships to 
their general question, and finally the highest priority 
items are presented with their means and rankings. 



TABLE 2 

Rankings and Means of General Questions 
(l=fnghest priority to 5=lowest priority) 



General Questions 



What Internal critical questions 
confront developmental programs? 
(6 parts) 

What criteria are appropriate for 
evaluation of developmental programs? 
(6 parts) 

What external questions confront 
developmental programs? (6 parts) 

What are the justificatio ns for 
a developmental program? T8 parts) 



Research and 
Evaluation 
Committee 
N=22 

janl ving x 



1 2.17 

2 2.23 

3 2.40 

4 2.73 



Executive Combined 
Commi tvee N=28 
N=6 



Ranking x Ranking x 



2.06 1 2.15 



3 2.18 2 2.22 



1 1.83 3 2.28 



4 2.25 4 2.63 
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Table 2 shows the rankings and means of, the general 
questions. This table reveals three striking aspects of 
this survey. The general question on criteria for 

evaluation was included primarily to gather information for 
another project^ yet the respondents gave it very high 
priority. The overall second ranking mandated this question 
and its associated items not be excluded from the analysis. 
Nextf there was a real difference in the rankings given by 
the two groups in the sample. Both the R&E respondents and 
the Executive Committee gave the justification question area 
least priority; apparently professionals feel less need to 
justify developmental programs. Both groups put internal 
questionis ahead of evaluation criteria as priority areas. 
The Executive Committee^ however ^ saw external concerns as 
much more important than did R&E members. Apparently the 
Executive Committee was more oriented to concerns outside 
the institution while R&E members showed more concern for 
internal concerns. Finally, an examination of the means 
showed the Executive Committee consistently recording lower 
means; this indicates a generally higher priority given to 
all items. 

Table 3 shows the ranked items arranged by their 
general question. Three of the four priority ranked items 
(If 3f and 4) came under the general question on internal 
critical questions. Items ^ and 4 both relate to concerns 
for program quality; they ai.3 ^ were the only ranked items 
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with a strong positive corr.elation . Although they relate to 
the same concern and were perceived that way in the aurvey, 
they still were given first and fourth priority among the 
twenty-six items rated. Under external concerns financial 
concerns were dominant and es^Jecially foe the Executive 
Committee members. 

Table four shows the priority ranking among the items 
on the survey. Critical questions of highest priority 
related to questions of effectiveness of developmental 
programs and their effect upon overall curricu?.u-? quality 
(Rankings 2, 4, 9 and probably 1), to public policy concerns 
generally associated with agencies external to che coj.lege 
(Rankings 5, 6, 7, 8 and 10) and to inst ? tut-ional 
priorities (Ranking 3 and possibly 1 and 10). 
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TABLE 3 

Ten Highest Priority Items by Ranked General Questions 
(Items number by ranking by all respondents) 

• 

First Ranked General Question 

What internal cr:'"?rii questions confront developmental programs? 

1. Do developmental courses or programs iftiprove basic skills? 

3. Do institutional .goals and priorities support a developmental 
program? 

4. Does having a developmental program positively effect curriculum 
quality? 

Second Ranked General Question 

What criteria are appropriate for evaluation of developmental programs? 

2. Persistence in college beyond the developmental program. 
_ 9. Basic skill improvement measured by pre- and post-tests. 

Third Ranked General Question 

What external questions confront developmental programs? 

6. Should students without academic skills be admitted to college? 
8. Should public monies pay for developmental education? 

10. Should developmental programs be cost-effective? 

Fourth Ranked General Question 

What are the justifications for a developmental program? 

5. Insuring academic basic skills training is available for all 
students. 

7. Providing a service to insure access to higher education for 
minorities, adults, etc. 

(Numbcjrs represent items as ranked on priority by all respondents - N=28.) 
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TABLE 4 



The Items Rated "Above Average" Priority by 
An Respondents with Mean and Rank. 
(Represents 10 of 26 items) 

ITEM MEAN RANK 



3.2 Do developmental courses or programs 1.28 1 

improve basic skills? 

4.2 An appropriate evaluation criteria 1.5C 2 
is persistence after the developmental 

program. 

3.4 Do institutional goals and priorities 1.62 3 
support a developmental program? . 

3.5 Does having a developmental program 1.72 4 
positively effect curriculum quality? 

1.3 Insuring academic basic skills training 1.746 5 
is available for all students justifies 

a developmental program. 

2.1 Should students without academic 1.75 6 
basic skills be admitted to college? 

1.2 Providing a service to insure access to 1.78 7 
higher education for minorities, adults, 

etc. justifies a developmental program. 

2.2 Should public monies pay for developmental 1.817 8 

education? 

4.6 An appropriate evaluationVriteria is basic 1.819 9 
skills improvement measured by pre- and 

post-tests. 

2.5 Should developmental programs be cost 2.04 10 

effective? 



NOTE: The criteria for inclusion was a combined item mean of 2.0 or less where 
2.0 represents "above average priority." 
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The data collected indicate that concerns for academic 
quality throughout academe are clearly present among 
developmental educators. The general questions used may 
have been mildly misleading. Although internally oriented 
critical questions were given the highest priority f a 
careful inspection of the ranked item responses suggests 
another interpretation. It appears that external agencies - 
usually state governing or coordinating bodies - are the 
source of most critical questions now facing developmental 
education. Several items under internal questions could as 
well relate to questions from external agencies ; this 
interpretation of the data would also be consistent with the 
highest priority on external questions expressed by the 
Executive Committee. Concerns for program quality, overall 
curricular integrityr proper evaluation criteria, and 
financial concerns all represent policy issues at a level 
external to the devel r)[ :Tiental program. These are state 
agency and governin9 I concerns for public institutions 

and governing boc*! ; \cerns for private institutions . ^ 
Faculty and administracuis within institutions also have 
strong interests and input in these areas, although my 
reading of this survey suggests a concentration of concern 
external to the institution. 

Critical questions internal to the institution involve * 
program evaluation and especially a concern for persistence 
in college beyond the developmental program and for cost 
effectiveness. Justifying a program within the institution 
is not very high priority. Where providing access has been 
accepted as part of ah 'institution' s mission^ developmental 
programs need little further justification. There is some 
indication, however, that for institutions ambiguous about 
their role in stimulating access to high education, 
justification is an ongoing need. 

The research agenda for an institution must relate to 
an assessment of internal and external questions. Does the 
institution have a clear mission? Is that mission 
consistent with a developmental program? What are 
acceptable criteria at that institution for a "successful" 
developmental program? Given those criteria, is the 
existing program successful? If not, how can it become so? 
What is the impact of the developmental program on the total 
curriculum? 

The research agenda for an organization like NADE is 
staggering. Public policy issues most dominate. Studies of 
state and federal policies affecting developmental education 
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should be conducted and disseminated to better determine 
where we are. Coordinated evaluation and cost effectiveness 
studies should be conducted with multiple institutions and 
states to provide base line data on program effects and 
costs. Models for evaluation ought continue to be developed 
and improved. An active progrc'm to identify information 
needed to inform policy debates related to developmental 
education should be made and appropriate follow-up studies 
conducted. . Finally, an improved survey might ba 
administered periodically to the NADE membership to monitor 
on-going membership perceptions of priority items and 
questions . 
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APPENDIX A 



.NATIONAL ASSOCIATION FOR 



• ^■LQPMENTAL EDUCATION A 
F .ese( i.-{r ti and Evaluation Committee 



SURVEY I J' 



Question One: What are the justifi- 
cations for a develop- 
' • . mental program? 

1. Protecting the quality of the 
mainstream curriculum. 

2. Providing a service to ensure 
access to higher education for 
minorities, adults, etc. 

3. Insuring academic basic skills 
training is available for all 
students. 

4. Providing an umbrella organi- 
zation for academic support 
services. 

5. Providing an umbrella for 
institution v/ide testing and 
setting minimum standards for 
entry to the instructional 
program. 

6. Providing socialization and a 
gradual introduction to the 
college community. 

7. Providing older students v/ith 
"refresher" work on basic 
skills, 

8. Providing a support to job 
training programs. 

9. 



10. 



1 2 



1 



Question Two: VJhat external critical 
questions confront 
development programs? 

1. Should students without academic 1 
basic skills be admitted to 
college? 

2. Should public monies pay for 1 
* development education? 

3. Should there be admissions 1 
criteria for developmental 
programs? 

4. Should developmental programs 1 
have mandatory program evalua- 
tions? 

5. Should developmental program 1 
be cost effective? 

6. Should external agencies 1 
determine the purpose and 

extent of developmental 
education? 



Question Three: What internal critical 
•questions confront 
developmental programs? 



7. 



1 



8. 



1 



1. 



Do students enter the college 
with appropriate academic basic 
skills? 



1 



2. 



Do developmental courses or 
programs improve basic skills? 



1 



3. 



Do tuition, fees, and govern- 
ment funding cover the costs 
of the developmental program? 



1 



Do institutional goals and 
priorities support a develop- 
mental program? 



1 
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5. Does having a developmental 
program positively effect 
curriculum quality? 

6. What students do the develop- 
mental program serve? Are they 
special interest groups? 

7. 



8. 



Question Four: What criteria are 

appropriate for evalu- 
ation of developmental 
programs? 

1. Persistence in the develop- 
mental program or course. 

2. Persistence in college beyond 
the developmental program. 

3. Persistence to degree or 
. program completion. 

4. Personnel development measured 
by completion of developmental 
courses. 

5» Cost-effectiveness.' 

6. Basic skill improvement 
measured by pre and post- 
tests. 

1. 
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SPEAKER/CONSULTANTS IN THE FIELD OF 
DEVELOPMENTAL EDUCATION 

by Sr. Mary Pardy, Ph.D. 

Copiah-Lincoln Jr. College 




AMA HSSESSHEHT/PLACm 



m 


SCHOOL 




ofELlAL TOriC 


Mn. Ann Poore 


GreeovlUe Tech. College 


Box S616, Eta. B, Greenville, S.C. 












Dr< Strah Sanderi 


U.S.C, 


Tnnfl 9tM Uftlfl 
iUUil r(Uj{i VOLB 


Dlagnoalng oral Engllah 


Dee Janee 




ABUi Nilit 


for TOEFL 




Anil. N 


MorUng with m. black 










Lee Noel 


ACT Progru 


lova City 




BhAroQ llayenger 


Hlnnaaoolli CoaiiuiilCv (!o11i>ffA 


ISOl Hennepin Ave^ 




iok & Karen Nleiaeyer 






VAhfltflf lln(vpriilfv 


470 E. Lnckuood 


ASflessflienc la nore Than 


Horrls HASsey 




St. louls, Ho. 


Teatlng 


University of Colorado 




Generation Value Cap 


David Durhaa 


Hinds Jc* College 


Raynond, HS. 39154 


Placing students with the 


Hr» Jdiaee Baber 






Cotaputer 

AsaeseQeat/FlaGenent 


Copiah'Linclon Jr. College 


Natchet, KS 39120 
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mA n ■ COUHSEIIHC h ADVISmC/mEMEHTIDN 



m 


SCHOOl 


ADDRESS 


Aleundri Kripsli 


USC/Coli 


TOEFL 


Ann Appleton 


Greenville lEC 


Box M, Sti. B, Greenville, S.C. 


BItubith BUchieU 


Greenville TEG 


Box M, Sti. B, Greenville) S.C. 

29646 


Bevtrly K. Hlcheal 


Univ. of Pittsburgh 


Celeitli Friction 
Karln NlcMyet 
Lucy Hiufjct Calvin, S.L. 
Jihp O'kelUv 
Hunur Boylm 


Hinneapolii Com. College 
Uabiter Univiriity 
Uebiter Univeriity 
yebiter Univeraity 
Uinde Jr. College 


1501 Hennepin Ave. 
HinneapollB, Hn. SS40] 
WO E. Lockvood 

St. LOUll, KO. (Ill; 

470 E. Lockvood 
St. Uuii, Ho. i}ll} 
470 E. Lockvood 

St. uuii, Ho. am 

Rayiond, Hs. 391S4 


Divld Diy« 
Hr. hm Baler 


Jeffecaon Divia Cmpus 
Copiih-LlncolD Jr. College 


HS Gulf GoBit Jr. College 
Gulf port, Ha. 39S01 
Natchet, Ha. 39120 



SPECIAL TOPIC 



Hith Anxiety 



An Acideilc li :arvention 
Progria 

Foreign Student Adv/C Early 
Virning of Acad. Difficulty 
Perional Couuellng/lnter. 
Alcotwl, Drugi, Anerexla, Deprei. 



CouQieling & Advising/Intervention 
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MBA 111 • DevelopBSDtal Currlcuhm (any national aapect) 



m 


SCHOOL 


ADDRESS 


LoU BoUoan 


HlnneapoUa Conmunlty College 


ISOl Hennepin Ave. 


inn fn/llALf 




MlnneapoUa, Kn. SS403 


Minneapolis Conuiuiilty College 


ISO! Hennepin Ave. 


WW] nunpnuvD 




HlaneapoUs, Mn. 5540] 


HlnneapolU Comiinlcy College 


1501 Hennlpk Ave. 


Hugh YasaiiotQ 




Minneapolis, Hn, 55403 


MlnneepoUe CosunuQUy Collese 


1501 UennlDln Ave. 


Diane Chambeifl 




HlaneapoUs, Mn. 55403 


IIq1v« of Hlnneeota 


Duluth, Minnesota 




St. Cloud State Ualverslty 


SC. Cloudi Nlnnesota 


Jobo ( Karln Nleaeyer 


Uebfltef Unlveriiltr 




JaneH Braafleld 




St. Loula, Ho. (3119 


Webster Unlveralty 


470 E Lockvood 


John Rouche 




St. Louis, Ho. 63119 


llnlveralty of Texas 


Austin, Texaa 


J.U. Canlchael, Jr. & 


Xavler Unlveralty 


Hew Orleans, LA 


Lester Jonea 


Dr. Hlctiiel iioban 


LaGuardla Coimunlty College 


31-10 Tooson Ave. 






Long Island, Nlf lllO: 



SPECIAL TOPIC 



HithaaeticB 

Science & Frobleu Solving 



Reading h Coaputer Soft 

The Expanding Role of the 
College Reading Teacher 
Coiputer Training for Faculty 
And Advisors 



Teaching Critical Read. & Anal. 
Reasoning In Project SOAR 
Develop Currlculun Design 



3. i I. fi4 



ERJC 



Am IV • THE LEARNER 



NAME 

Dr, B«U Uerloog 
Dr. Uci Duoa 
k, Joieph Uedvehouie 
Uuia yelul 
Chair. HAUki 
Sutin Leflar 
Dr, Dilo Jordan 
George Spear 
Dr. Phil Carter 
Ualcer Pauk 
Bernlce HeCarthy 
Elaine Duncan 
Dr. Corlnne Anderaoit 



SCHOOl 
Chair. Eng. Dept. 
St. John's Univ. 
A.S.I). 

AuguBtana College 

Jordan-Adam learning Ctr, 

UHKC 

K-State 

Cornell UnlveruUy 

Jefferaon Davis CiipuB 
Coplah'Llncolo Jr. College 



ADDEESS 

Salenla High Schoc' 
Salinda, S.C. 29US 
Jauca, N.Y. 

Foooe. N.C. 

Oh^ 

F 'C; IiUal, 111. 



WOO H. Fortlao 
Oklahoia City, Ok. 73112 
School of Education 52 Cherry 
Kaniai City, Ho. 
Hanhatten, Ki, 



Suite 101, 6-:0 Enterprlie Dr. 
Oak Brook,, i:, 60521 
HS GuU Ccaat Jr. College 
Gulfport, Ha. 39501 
VeoaoD, Ha 39191 



SPECIAL TOPIC 

Teachlng/Llt. k vrltlng 
to Rea/itudeati 
Learning Stylai 

Adult Lecroeri 

Adult Learn/dla 
(London Procedure) 



Learning DliablUtlei 



"Developnent of the retired 
learner" 



4Kat Systeq 
Learning Styles 
The Adult Leorner or 
or Dlveralty Heetlng Needa 
Planning, lapleaentatlon and 
Evaluation (P.I.E.) of Learner 
Keedi 
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AXEA V - TUTORIAL SEHVICES 



HfilE SCIlOOt fflSSi SPECIAL TOPICS 

lltt.lottle Glbion Greenville Tec. Cclhge S'"* Greenville, S.C. 

Or. Bottle Uorne Under CoUeiie Greenwood, S.C, 256*6 Linguistic techniques 

forworkof reD/etudent 

BerteBtflwn Unlv, Cal/ San Diego San Diego, CA Tutor Tr. 

JohD Vhltney Hilton Univ. Of lUonoia 8 ^\imih> lUlnola peer Tutors 

Barbara E. Broun HlghUnd Coaa, College (Haven'a heard her apeak, but 

she has a good prograo) 

Celeotls Fractloa HlnneapoUe Connlty College ,, 

1501 Hennepin Ave. Training i Set-up Tutovlal Svca. 
Rarln Nleiieyar Vebstet University HlnneapoUs, Hn. 55*03 

470 £. Lockwood The Senior Tutoring Service 
Don EbyfiO K.C. Ks Coaiaunlty College "'1' 

7150 State Ave. Conputerhlng and coordinating a 

K'C.iKa 65112 tutoring network 
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Am VI ■ LEAMIHC CEHTERS 



M SCHOOL ADDRESS SPECIAL TOPIC 

Giilfotd Tech. CoUege Hudnon, H.C. Leacnlnj Center 

Trlccn College uUnolB 

Helnl Dade Cobii. CqI. Florida (Don't have a perion In lalnd 

. juit heard progrBn ia looi.) 

HarthaHaxwell 

^""^ Longbeach, CA 

CelGntU Fraction HlnneapolU CoDflunliy College 1501 Hennepin Ave. 

HloneipoUi, Kn. 

^^^^^ HlnneapolU Comaunlty College 1501 Hennepin Ave. 

Hlnneapollf, Hn. 5540] 

John Nleaeyer Vebater Unlverglty 470 E, Lockwood What la a Learning Center? 

St. Louli, Ho. 63119 



6. 



AREA VII - Staff Developaent 



SCHOOL 



ADDRESS 



SPECIAL TOPIC 



Dr. Hatty Uerbeit 


Greenville Tec. 


He. Linda Forrester 


Greenville Tec, 


Curtli Klles 


Fiednsnt Tec. College 


Hunter Boylan 




Elicatlon 


Johfl & Karln Nleneyer 


Uebster Unlvecelcy 


Janes Brasfleld 


Uebster University 


HllUe Colling 


Jef fereon Davla Caiopus 


Staff (ck. vith Dr. Hlchael Uobai 


a) UGuardia Com^ College 



Box 5616 Sta. B. Greenville, S.C. 

29646 

Box 5616 Sta. B. Greenville, S.C. 

29646 

F.O, Drawer 1467 Greenwood, S.C. 

29646 

Appalachian State Univ. Booae, NG 
28607 

470 E. Lockwood 
St. LouU, Ho, 63119 
470 E. Lockwood 
St. Louis, Ho. 63119 
Gulf port, Hs. 39S01 

31*10 Thomson Ave. 
Long, Island, N.Y. lllOl 



Problem Solving, Reading 

Learning Ctra. Data CoUecdon 
& Reaaerch 

The Expand Ifig Role of the College 
Reading Teacher 

Computer Training for Faculty h 
Advliors 

The Role of Staff Development within 



Integrated Skills Eelnforcenent (IRS) 
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OTHER 




Computer Software & Evaluation of Computer Software 

1. Elmer Matllla, Minneapolis Community College, 1501 Hennepin Ave. 

Minneapolis, MN. 55403 

2. Tom Boe, Minnesota Educational Computing Consortium, 3490 Lexington 

Ave. Arden Hills, Mn. 55112 

3. John Nlemeyer, Webster University, 470 E. Lockwood, St. Louis Mo. 63119 
Management: The Moving Force in Developmental Education 

Group seminars for the disoriented learner 

Dr. Mildred Steele, Central College, Pella, Iowa 50219 

A Longitudinal Study of College Seniors Who had Below-;.\'erage Academic 

Records as Freshmen 

Melinda Hartley Southern Univ., New Orleans, LA Challenges Facing Development 



( 



ERIC Clearinghouse for 
Junior Colleges 
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